A comparison of performance and satisfaction between two types of group decision support systems

Group Decision Support Systems (GDSSs) and other electronic meeting technologies have been developed to support or replace traditional, verbal meetings. While extensive research has been conducted regarding the impact of these systems on the group decision making process, the vast majority of these studies have focused on groups meeting fact-to-face in one room. This paper focuses on how group members perform when distributed as non-proximate sub-groups (virtual legislative sessions) as compared to proximate, face-to-face groups (synchronous leg­ islative sessions). Experiments involving 12 groups of 10 members each showed that there were few significant differences in productivity and satisfaction between the two types of meeting formats. These and other results indicate that groups may operate productively in a virtual meeting environment. INTRODUCTION Group Decision Support Systems (GDSSs) and other electronic meeting technologies have had a positive impact on group communication and decision making by providing participants with an anonymous electronic forum for the simultaneous exchange of ideas and preferences (Dennis, George, Jessup, Nunamaker, & Vogel, 1988). However, most meetings using these sys­ tems have been conducted in a face-to-face meeting environment. It is now technologically fea­ sible to create facilities that will allow remote groups to interact as an intact virtual group in real time; that is, while sub-groups are face-to-face at a specific geographic location, the group as a whole may be comprised of distributed sub-groups (Nunamaker, Briggs, & Romano, 1993). 31 1 Vanjani et al.: A comparison of performance and satisfaction between two types of Published by CSUSB ScholarWorks, 1997 Journal of International InfonriatiQn Management Volume 6, Nuniber 2 One project that seeks to create rneeting facilities fpr virtual groups is currently underway at the University of Arizona. The Mirror Project is ". . . designed to cornbine different commu­ nication channels in an environment that integrates audio, visual, and textual media to provide synergistic, comprehensive, and robust support for effective and efficient group interaction" (Chappell, Vogel, & Roberts, 1992). In addition, the Computer-Aidefj Rusipess Engineering (CABE) project at the University of Arizona will create ineeting facilities that will allow geo­ graphicallyand temporally-distributed sub-groups at Air Force sites to be linked through a GDSS and teleconferencing. Very few studies have focused on distributed sub-group meetings, however. This research examines the differences between these distributed sub-group (or virtual legislative) meetings and traditional face-to-face (or synchronous legislative) meetings (DeSanctis & Gallupe, 1987). Results of the study show that very few significant differences between the two types of meetings exist. Therefore, groups may be able to meet efficiently and effectively in distributed, sub-group environments. PRIOR RESEARCH Previous GDSS research generally has concentrated on small and large groups meeting in a non-distributed or face-to-face environment, but some research has been undertaken to com­ pare face-to-face meetings with nominal groups (groups in which members work individually at separate locations). In one such study (Valacich, George, Nunamaker, & Vogel, 1990), research­ ers found that the nominal groups were more productive than the face-to-face (FTP) groups (as measured by idea quantity and qualitj;, and nominal group members were more productive per person. Members of four-person nominal groups were the most productive and eight-person FTF groups were the least. The researchers attempted to explain this difference by pointing out the fact that more verbal interruptions occurred in the FTF groups (laughing, talking about a written comment, etc.). There was no significant difference in satisfaction measures between the FTF and nominal groups, however. In another study involving sixand 12-member nominal (non-communicating) and FTF (communicating) groups (Dennis & Valacich, 1993), researchers found that there were no differ­ ences in the six-member groups, but the 12-member FTF groups generated more ideas than did the 12-member nominal groups. These results were somewhat confirmed in a later study in which researchers found that FTF groups generated more ideas with higher quality than did groups of physically-separated individuals or sub-groups which pooled their comments after the meeting (Dennis & Valacich, 1994). Another study (Valacich, George, Nunamaker, & Vogel, 1994) measured the idea genera­ tion performance of groups of four and eight members in both proximate and distributed condi­ tions using the same synchronous computer-mediated communication systems. The distributed groups outperformed the face-to-face groups in terms of the total number of unique, quality, and high-quality ideas generated. 32 2 Journal of International Information Management, Vol. 6 [1997], Iss. 2, Art. 3 http://scholarworks.lib.csusb.edu/jiim/vol6/iss2/3 A Comparison of Performance Journal of International Information Management Another pilot study compared face-to-face groups with dispersed-synchronous and dispersed-asynchronous groups (Burke & Chidambaram, 1995). Face-to-face groups experienced more effective leadership and coordination competence as compared to the distributed groups. There was no difference between the members of any group type regarding task performance and perceptions of cohesiveness and equality of participation. These results led the authors to con­ clude that electronically-distributed work groups can become cohesive and perform effectively, provided that all the group members have adequate training and sufficient time for the meeting. In perhaps the only prior study of synchronous legislative and virtual legislative groups (Aiken & Vanjani, 1996), virtual groups of eight people each wrote more quality comments, thought the comments were more anonymous, were more satisfied with the system, and believed participation among their group members was more uniform. However, there were no significant differences between the two types of groups in terms of idea satisfaction. Moreover, group mem­ bers in both types of meetings did not think it was important to be able to see each other during the electronic discussion. RESEARCH METHODOLOGY To confirm the results of earlier research and investigate additional group process and outcome variables, this study was conducted using the synchronous or face-to-face (FTP) and virtual or non-face-to-face (NFTF) legislative session meeting formats. Subjects Six groups of 10 undergraduate business students each met in the FTP environment, and six groups of 10 met in the NFTF environment (each of these six groups split into two sub­ groups). The students received extra credit for their participation. Procedures Each subject was assigned to one of the two groups and was briefed regarding the meeting structure. Next, a 10-minute warm-up session allowed the subjects to acquaint themselves with the software and technology. The actual meeting took 10 minutes. A creative task for idea genera­ tion was used in the experiment. The subjects were asked to write as many comments about the solution of the parking problem on campus as they could. Dependent Variables and Measurement Following the meeting, subjects completed a self-assessment questionnaire which asked them to rate on five-point Likert scales their opinions of several facets of the meeting. In addition, the number of comments generated by each group was recorded. A quality comment was defined as a comment that pertained to the discussion topic. A unique, quality comment was defined as a comment related to the discussion and mentioned for the first time during the meeting. Two independent raters reviewed the transcripts to ascertain the number of quality and unique, quality comments. 33 3 Vanjani et al.: A comparison of performance and satisfaction between two types of Published by CSUSB ScholarWorks, 1997 Journal of International Information Management Volurne 6, Number 2 EXPERIMENTAL RESULTS Group Statistics The comments generated during each of the meetings were reviewed by two raters, and inter-rater reliability tests indicated a high degree of agreement between tliem (jCronbach Alpha = 0.99). There were no significant differences between the FTP and NFTF groups in terms of the numbers of raw (F = 2.57, p = . 14), quality (F = 0.64, p .44), and unique, quality (F = .44, p = .52) comments. Table 1. Questionnaire Summary for ALL Resppndents (see the Appendix) Group Type ALL (N = 120) Variable: Mean Std. Dev. Satisfaction with member proximity 3.Q3 1.31 Perception of group cohesiveness 3.04 1.21 Ease of communication 2.35** 1.26 Perceived anonymity 2.04** 0.95 Satisfaction with ideas generated 3.43** 1.07 Satisfaction with the system 4.21** 1.01 Preference to see all participants FTF 2.49** 1.30 Perception of production blocking 1.75** 1.00 Evaluation apprehension 3.91** 1.22 Perception of free riding 3.67** 1.31 Satisfaction with group membership 3.84** 1.08 Perception of group member participation 3.74 1.03 Time sufficiency for meeting 2.11** 1.40 ** Significantly different from 3.00 at alpha = 0.05 34 4 Journal of International Information Management, Vol. 6 [1997], Iss. 2, Art. 3 http://scholarworks.lib.csusb.edu/jiim/vol6/iss2/3 A Comparison of Performance Journal of International Information Management Table 2. Questionnaire Summary for FTF Respondents (see the Appendix) Group lype FTF (N = 60) Variable: Mean Std. Dev. Satisfaction with member proximity 2.80 1.36 Perception of group cohesiveness 3.13 1.23 Ease of communication 2.27** 1.30 Perceived anonymity 2.15** 0.94 Satisfaction with ideas generated 3.42** 1.20 Satisfaction with the system 4.27** 1.02 F*reference to see all participants FTF 2.52** 1.28 Perception of production blocking 1.63** 0.94 Evaluation apprehension 3.67** 1.35 Perception of free riding 3.47** 1.31 Satisfaction with group membership 3.87** 1.19 Perception of group member participation 3.77** 1.08 Time sufficiency for meeting 2.18** 1.44 ** significantly different from 3.00 at alpha = 0.05 Table 3. Questionnaire Summary for NFTF Respondents (see the Appendix) Group Type NFTF (N = 60) Variable: Mean Std. Dev. Satisfaction with member proximity 3.27* 1.22 Perception of group cohesiveness 2.95 1.20 Ease of communication 2.43** 1.23 Perceived anonymity 1.93 * * 0.95 Satisfaction with ideas generated 3.45** 0.93 Satisfaction with the system 4.15** 1.01 Preference to see all participants FTF 2.47** 1.32 Perception of production blocking 1.87** 1.05 Evaluation apprehension 4.15** 1.04 Perception of free riding 3.87** 1.29 Satisfaction with group membership 3.82** 0.97 Perception of group member participation 3.72** 1.03 Time sufficiency for meeting 2.03** 1.23 ** significantly different from 3.00 at alpha = 0.05 * significantly different from 3.00 at alpha = 0.1 35 5 Vanjani et al.: A comparison of performance and satisfaction between two types of Published by CSUSB ScholarWorks, 1997 Journal of International Infonnation Management Volume 6, Number 2 Individual Statistics Tables I, 2, and 3 contain the means and standard deviations from the questionnaire re­ sponses. The tables also report significant differences in the overall response to each of the questions from the median value of 3 on the Likert scales. Analyses of variance tests were also conducted on the evaluations. There was a statistically significant difference in the responses (at the a = .10 level) between the FTP groups and the NFTF groups for only three variables: satisfaction with member proximity (F = 3.91, p = .05), evaluation apprehension (F = 4.84, p = .03), and perception of free riding (F = 2.83, p = .09). These and other results are discussed below. Satisfaction with Member Proximity. The mean response for the FTP participants was not significantly different from 3 (2.80), and the mean response for all participants also was not significantly different from 3 (3.03). However, the mean response for NFTF participants was significantly higher than 3 (3.27). This implies that the NFTF participants did not consider group member proximity as being important for their satisfaction with the meeting process. About 67% of all participants (53% of FTP and 80% of NFTF participants) were either indifferent to or did not consider group member proximity important for their satisfaction. Thus, a lack of group member proximity may not be an inhibiting factor vis-a-vis satisfaction with the meeting. Group Cohesiveness. There was no significant difference between the types of group mem­ bers regarding perception of group cohesiveness. The mean response for the perception of group cohesiveness was 3.13 for FTP groups, 2.95 for NFTF groups, and 3.04 for all respondents. About 59% of all the participants (57% of FTP and 62% of NFTF participants) felt that group member proximity was not very important for feelings of group cohesiveness. Communication Ease with Group Members. There was no significant difference in ease of communication perceptions among the two types of groups. The mean response for ease of com­ munication was 2.27 for FTP group participants, 2.43 for NFTF group participants, and 2.35 overall for all participants. Most respondents felt that it was relatively easy to communicate with group members using the system. About 80.0% of all respondents (82% of FTP and 78% of NFTF participants) were either satisfied or indifferent regarding communication ease with re­ spect to group members. Anonymity. Although no names were attached to comments, anonymity may be reduced if group members are able to see each others' screens. However, there was no significant difference in anonymity perceptions between the two types of groups. The respective mean values for per­ ception of anonymity were 2.15, 1.93, and 2.04 for FTP groups, NFTF groups, and all respon­ dents. The frequency distribution shows that 90.8% of all respondents agreed that anonymity was reasonably protected. This percentage was 90.0% for the FTP group members and 91.7% for the distributed group members. Idea Satisfaction. There was no significant difference between the two types of groups regarding satisfaction with the ideas generated. The mean response was 3.42 for the FTP groups, 3.45 for the NFTF groups, and 3.43 for all respondents taken together. The frequency dis­ 36 6 Journal of International Information Management, Vol. 6 [1997], Iss. 2, Art. 3 http://scholarworks.lib.csusb.edu/jiim/vol6/iss2/3 A Comparison of Performance Journal of International Information Management tribution shows that while 78.3% of the FTF participants were neutral or satisfied with the ideas proposed by their group, the corresponding figure for the NFTF groups was 85.0%. About 82% of all participants were neutral or satisfied. System Satisfaction. Most subjects were satisfied with the system with mean scores of 4.27 for FTF participants, 4.15 for NFTF participants, and 4.21 for all respondents. The frequency distribution is also fairly consistent as 91.7% of both FTF and NFTF (and consequently all participants) were neutral or satisfied with the system used. Preference for Face-to-Face Meetings. Neither of the two groups considered it important to be able to see all group members during the meeting. The mean response was 2.52 for FTF participants, 2.47 for NFTF participants, and 2.49 for all participants. The frequency distribu­ tion shows that overall, 75% of all respondents were either indifferent to or considered being able to see all members of their group unimportant. The respective figures for FTF and NFTF groups were 73.3% and 76.7%. Production Blocking. There was no significant difference between the two groups in regard to perceptions of communication ease. The mean response was 1.63 for FTF groups, 1.87 for NFTF groups, and 1.75 for all respondents. The frequency distribution also confirms this result as 90.8% of overall participants perceived low or no production blocking. The corresponding figure for the NFTF groups was slightly lower at 88.3%, and slightly higher for the FTF groups at 93.3%. Evaluation Apprehension. There was a significant difference between both groups regard­ ing feelings of evaluation apprehension: FTF groups were more apprehensive of peer opinion of their comments than the NFTF participants. The mean response for the NFTF group members was 4.84 while the mean response for FTF group members was 3.67. The overall mean response was 3.91. The frequency distribution shows that while 85.8& of all participants experienced low to no evaluation apprehension, the variation among groups was significant. In the case of the NFTF groups, the figure was as high as 91.7% compared to only 80.0% for the FTF groups. Free Riding. The perception of free riding was significantly higher for the NFTF group participants. The mean responses were 3.87 for NFTF participants, 3.47 for FTF participants, and 3.67 for all respondents. The frequency distribution confirms this result as 81.7% of the NFTF respondents perceived some degree of free riding as opposed to 76.7% of the FTF respon­ dents and 79.2% of all participants. Group Membership Satisfaction. There was no significant difference regarding satisfac­ tion with group membership between the two types of groups. The mean score was 3.87 for the FTF group members, 3.82 for NFTF group members, and 3.84 for all respondents. The fre­ quency distribution shows that 88.3% of the FTF participants were neutral or satisfied with their group membership as were 93.3% of the NFTF participants and 90.8% of all subjects. Participation. Most participants perceived that participation was fairly equal for their re­ spective groups. The mean response was 3.77 for FTF group members, 3.72 for NFTF group members, and 3.74 for all respondents. The frequency distribution also had similarly close 37 7 Vanjani et al.: A comparison of performance and satisfaction between two types of Published by CSUSB ScholarWorks, 1997 Journal of International Information Management Volume 6, Number 2 results as 90.8% of all respondents, 91.7% of FTF respondents, and 90.0% of NFTF respondents perceived that all group members participated equally in the discussion. Time Sufficiency. All meetings were allotted ten minutes and all groups discussed the same task using the same system. There was no significant difference between the two types of groups in perceptions of meeting time sufficiency. The mean score was 2.18 for FTF group members, 2.03 for NFTF group members, and 2.11 for all respondents. The frequency distribution shows that 78.0% of the FTF participants, 83.3% of the NFTF participants, and 80.8% of all partici­ pants felt that they had sufficient time for the meeting. CONCLUSIONS The objective of this study was to study differences between groups meeting face-to-faee (FTF) and not-face-to-face (NFTF). Productivity, as measured by the total number of comments generated, number of quality comments generated, and number of unique, quality comments generated was not significantly different between the two types of groups, but these measures were slightly higher for NFTF groups. Further, a lack of member proximity did not have a negative impact on group members' meeting satisfaction or group cohesiveness. NFTF group members did not consider group member proximity important for them to feel like a part of a larger group. All participants, FTF and NFTF, were satisfied with the ease of communication, and all meeting participants were generally satisfied with the ideas that their group members proposed. However, the percentage was higher for the NFTF group members. Neither the FTF participants nor the NFTF participants considered it important to be able to see everybody in their group. Although all participants used the same system, and all communication and interaction was anonymous, the NFTF participants had significantly lower evaluation apprehension. This may be attributed to the lack of physical proximity. It would be interesting to study the impact of the addition of audio and video support in future distributed meetings using virtual groups. Since most participants could not identify who was working, they did not perceive much free riding. However, the members of the NFTF groups indicated a slightly higher and statisti­ cally significant difference in perceived free riding. Generally, all groups will have some mem­ bers who will not participate. However, a lower perception of free riding should have a positive impact on those who are participating. All groups were equally satisfied with their group membership. However, although there was no statistically significant difference between the two meeting types, a higher number of NFTF group members were satisfied with being members of their group. This too is an important result for the future of virtual meetings. The implication is that if the system allows efficient communication, then the lack of physical proximity should not have a negative impact on group cohesiveness. 38 8 Journal of International Information Management, Vol. 6 [1997], Iss. 2, Art. 3 http://scholarworks.lib.csusb.edu/jiim/vol6/iss2/3 A Comparison of Performance Journal of International Information Management Almost all participants felt that their group members participated fairly equally. This shouldresult in positive feelings about the meeting and group membership. However, it is interesting tonote that there was very little opportunity for them to actually see which members of their groupwere participating. There was less opportunity for the NFTF group members, since they couldnot see the other half of their group. Even so, members of both groups reported equal participa­tion.These results imply that there is a good potential for success with virtual electronic meet­ings in the future. REFERENCESAiken, M. & Vanjani, M. (1996). A comparison of synchronous and virtual legislative sessiongroups faced with an idea generation task. Working paper. University of Mississippi. Burke, K. & Chidambaram, L. (1994). Development in electronically-supported groups: A pre­liminary longitudinal study of distributed and face-to-face meetings. Proceedings of theTwenty-Seventh Annual Hawaii International Conference on Systems Sciences, 4, 104-113. Chappell, D., Vogel, D., & Roberts, E. (1992). The Mirror project: A virtual meeting place.Proceedings of the Twenty-Fifth Annual Hawaii International Conference on systemsSciences, 4, 23-33. Dennis, A., George, J., Jessup, L., Nunamaker, J., & Vogel, D. (1988). Information technology tosupport electronic meetings. MIS Quarterly, 12(4), 591-624. Dennis, A. & Valacich, J. (1993). Computer brainstorms: More heads are better than one. Jour­nal of Applied Psychology, 78(4), 531-537. Dennis, A. & Valacich, J. (1994). Group, sub-group, and nominal group idea generation: Newrules for a new media? Journal of Management, 20(4), 723-736. DeSanctis, G. & Gallupe, R. (1987). A foundation for the study of group decision support sys­tems. Management Science, 33(5), 589-609. Nunamaker, J., Briggs, R., & Romano, N. (1933). Meeting environments of the future. Groupware1993 Proceedings (S&n; Mateo, CA). New York: Morgan Kaufman Publishers, 125-144. Valacich, J., George, J., Nunamaker, J., & Vogel, D. (1994). Physical proximity effects on com­puter-mediated group idea generation. Small Group Research, 25(1), 83-104. Valacich, J., George, J., Nunamaker, J., & Vogel, D. (1990). Supporting flexible organizations:Varying group size and proximity in an electronic meeting system. Working paper. Uni­versity of Arizona. 399Vanjani et al.: A comparison of performance and satisfaction between two types of Published by CSUSB ScholarWorks, 1997 Journal of International Information ManagementVolurne 6, Nurnber 2 APPENDIXExperimental Questionnaire1. Sex: Male Female 2. Classification: Freshman Sophomore Junior Senior